Economic impact of a machine learning-based strategy for preparation of blood products in brain tumor surgery

Background Globally, blood donation has been disturbed due to the pandemic. Consequently, the optimization of preoperative blood preparation should be a point of concern. Machine learning (ML) is one of the modern approaches that have been applied by physicians to help decision-making. The main objective of this study was to identify the cost differences of the ML-based strategy compared with other strategies in preoperative blood products preparation. A secondary objective was to compare the effectiveness indexes of blood products preparation among strategies. Methods The study utilized a retrospective cohort design conducted on brain tumor patients who had undergone surgery between January 2014 and December 2021. Overall data were divided into two cohorts. The first cohort was used for the development and deployment of the ML-based web application, while validation, comparison of the effectiveness indexes, and economic evaluation were performed using the second cohort. Therefore, the effectiveness indexes of blood preparation and cost difference were compared among the ML-based strategy, clinical trial-based strategy, and routine-based strategy. Results Over a 2-year period, the crossmatch to transfusion (C/T) ratio, transfusion probability (Tp), and transfusion index (Ti) of the ML-based strategy were 1.10, 57.0%, and 1.62, respectively, while the routine-based strategy had a C/T ratio of 4.67%, Tp of 27.9%%, and Ti of 0.79. The overall costs of blood products preparation among the ML-based strategy, clinical trial-based strategy, and routine-based strategy were 30, 061.56$, 57,313.92$, and 136,292.94$, respectively. From the cost difference between the ML-based strategy and routine-based strategy, we observed cost savings of 92,519.97$ (67.88%) for the 2-year period. Conclusion The ML-based strategy is one of the most effective strategies to balance the unnecessary workloads at blood banks and reduce the cost of unnecessary blood products preparation from low C/T ratio as well as high Tp and Ti. Further studies should be performed to confirm the generalizability and applicability of the ML-based strategy.


Introduction
Brain tumor surgery is one type of operation that presents the risk of intraoperative transfusions such as meningioma, suprasellar tumors, cerebellopontine angle tumors, and skull-based procedures [1][2][3]. However, the literature review showed excessive blood products preparation has been observed in neurosurgical operations, which was estimated from the high crossmatch to transfusion (C/T) ratio. Chotisukarat et al. reported a C/T ratio for brain tumor operation at 5.0 [2], while Saringcarinkul et al. reported C/T ratios for meningioma, suprasellar tumor, and cerebellopontine angle tumor operations at 4.0, 4.2, and 8.7, respectively [3]. Over-ordering of preoperative blood products preparation can lead to an unnecessarily increased workload at blood banks as well as expired blood products. Hence, several management approaches have been studied to enhance the effectiveness of blood transfusions, such as patient blood management programs for the early detection and proper management of preoperative anemia [4], optimization of hemoglobin levels for transfusion [5], and preoperative autologous donation [6].
The pandemic caused by the Coronavirus (COVID-19) has negatively affected blood donations, which have decreased globally to approximately half that of typical periods [7][8][9]. Hence, the effectiveness of the protocol for preoperative blood products preparation should be a concern in situations involving limited resources. There are several strategies used to calculate the Maximum Surgical Blood Order Schedule (MSBOS) as follows: Procedure-based guideline [10,11], 1.5 times the transfusion index [3,7], and routine protocol [12].
Nowadays, machine learning (ML) has been studied in clinical research to support decision-making [13,14]. However, how machine learning is used in general practice remains a challenge [15]. For example, an ML-based screening system for COVID-19 has been developed [16,17] and subsequently implemented in clinical practice via mobile phone [18]. Additionally, Tunthanathip et al. used a random forest algorithm to optimize cranial computed tomography in children after a traumatic brain injury via a web application [19,20].
The economic perspective is one of the outcomes that could be evaluated by ML implications because the over-requisition of preoperative blood products preparation has been reported in prior studies, which tends to increase the burdens on blood banks, as well as the wastage of blood products and unnecessary costs. Consequently, this research aimed to identify the cost differences of the ML-based strategy compared with other strategies in preoperative blood products preparation. Besides, a secondary objective was to compare the effectiveness indexes for blood products preparation among the strategies.

Study design and study population
The study utilized a retrospective cohort design conducted on patients who had been diagnosed with a brain tumor and undergone surgery between January 2014 and December 2021 at a single-center hospital. However, patients whose medical records contained incomplete transfusion data or who received a blood transfusion before surgery were excluded. Baseline clinical characteristics, preoperative hematologic laboratories, and operational data were collected from electronic-based medical records.

Strategies for preoperative blood products preparation
In the present study, three strategies for preoperative blood products preparation were compared as follows: ML-based strategy, clinical trial-based strategy, and routine-based strategy.

ML-based strategy
The first strategy for preoperative blood products preparation was developed by the training of the ML predictive model that forecasted the number of units of blood products from several datasets.
Total data in the present study comprised two major cohorts of data that corresponded with two processes of workflow as follows: 1) Development of predictive models and deployment of the ML-based web application, and 2) validation of the web application, as shown in Fig 1. The first cohort involved data from 1,267 patients diagnosed with brain tumors between January 2014 and December 2019, which was divided into a training dataset and a testing dataset using the 70:30 random splitting method. Consequently, 887 patients were included in the training dataset, while the testing dataset comprised 380 patients for intraoperative blood transfusion predictability. Details concerning the development and deployment of the MLbased web application are described in the Supplement.
The second cohort comprised data from 414 patients who had undergone brain tumor surgery between January 2020 and December 2021. Data for this cohort was used for validation of the predictive models that were built via the web application, as shown in Fig 2. The tool is simple to use by scanning the quick response (QR) code or getting the uniform resource locator (URL) on laptops or smartphones, after which the web application will be ready to use. The web application was developed and deployed for estimating the cost differences between strategies. Additionally, the results of web application validation are reported in the Supplement.

Clinical trial-based strategy
Transfusion protocol in neurosurgery varies across institutions; no standard guideline has been recommended for practice in neurosurgical operations from a prior systematic review [21]. For comparison, we referred to an analogous strategy from previous clinical trials [22,23] that mentioned the number of packed red cell (PRC) and FFP units based on the FFP: PRC ratio of 1:1.

Routine-based strategy
A routine-based strategy based on routine crossmatch at our institute was used as a reference group for comparison. A summary of preoperative blood products preparation for each strategy is shown in Table 1. Moreover, the cost per unit of blood products preparation for comparison among strategies is presented in Table 2.

Operational definitions of the effectiveness indexes
According to the secondary objective, a comparison of the effectiveness indexes among strategies was performed; these indexes are defined as follows: [24] C/T ratio is defined as the number of units crossmatched/number of units transfused, with a C/T ratio of 2 or less indicating effective blood utilization [25,26].
Transfusion probability (Tp) is defined as the total number of patients transfused/total number of patients cross-matched × 100. A Tp of 30% and above indicates effective blood usage [24,26].
Transfusion index (Ti) is defined as the number of units transfused/number of patients cross-matched. A Ti of 0.5 or more indicates effective blood usage [24][25][26].

Statistical analysis
Categorical factors were presented as frequencies and percentages using descriptive statistics, whereas continuous factors were performed by mean and standard deviation.
Using data from the second cohort, the comparison of cost and effectiveness indexes among strategies was analyzed. In detail, the routine-based strategy concerned the reference group, and independent t-tests were performed to compare the mean between groups. Therefore, a p-value < 0.05 was determined as statistically significant. Statistical analysis was performed using R version 4.0.5 (The R Foundation for Statistical Computing; Vienna, Austria). The two independent means test formula was used as well as for sample size calculation [27]. From the study of Alghamdi et al. [25], a comparison of the cost difference between a prior crossmatch strategy and a new strategy reported figures of 55,560$ and 43,316$. Moreover, we defined the acceptable standard deviation for both groups at 35,000$. Therefore, the minimum total sample size needed to test the hypothesis was 387 patients.

Ethical considerations
The human research ethics committee of Faculty of Medicine, Prince of Songkla University approved this research (REC 64-477-10-1). Informed consent from patients was not required due to the nature of the retrospective study design. However, patients' identification numbers were encoded before analysis.

Results
The patients' baseline characteristics and preoperative laboratories for the second cohort are presented in Table 3. The mean age was 46.36 ± 17.3 years, and the majority of patients were female (57.5%). The major underlying diseases were hypertension, dyslipidemia, and diabetes mellitus. Moreover, a preoperative seizure was observed in 10.6% of cases. Of all cases, 38.2% were meningioma, while glioma and pituitary adenomas were 30.7% and 11.4%, respectively. The mean tumor diameter was 3.13 ± 0.90, whereas the mean preoperative midline shift was 0.45 ± 0.30. By operation, more than half of all surgeries involved craniotomy with tumor removal, while the endoscopic transsphenoidal approach was performed in 12.3% of cases.
Among strategies with a 2-year time period, C/T ratio, Tp, and Ti are established in Table 4. The ML-based strategy had the lowest values for all effectiveness indexes compared to other strategies, while the routine-based strategy demonstrated a high C/T ratio and Tp that were over the reference for effective blood utilization. When comparing the indexes among strategies with quarters of the period, the mean difference of all indexes between the ML-based strategy and routine-based strategy was potentially significant, as shown in Fig 3 (p<0.001). Moreover, the mean difference of the C/T ratio and Tp between the ML-based strategy and clinical trial-based strategy was statistically significant (p 0.001 and p<0.001, respectively).
The cost and cost differences among strategies were calculated to estimate the economic impact. The overall costs of blood products preparation among the ML-based strategy, clinical trial-based strategy, and routine-based strategy were 3,0061.56$, 57,313.92$, and 136,292.94$, respectively. From the cost difference between the ML-based strategy and routine-based strategy, cost savings of 92,519.97$ (67.88%) were apparent when implementing the ML-based strategy instead of the routine-based strategy in the practice for the 2-year period. In addition, cost savings of 47.88% were observed when the ML-based strategy was implemented instead of the clinical trial-based strategy, as shown in Fig 4.

Discussion
The results of the present study observed an imbalance between preoperative blood products preparation and actual utilization intraoperatively in brain tumor surgery over a 2-year period, in concordance with other research reports. Saringcarinkul et al. [2] studied the effectiveness indexes of 377 patients who had undergone neurosurgical operations and found that almost all brain tumor surgeries had C/T ratios over the effectiveness threshold, whereas Chotisukarat et al. [3] found C/T ratios for brain tumor operations ranged from 5-12 and Tp ranged from 7-20%. In routine practice, neurosurgeons typically request more units of preoperative blood products for safety in cases of unexpected bleeding intraoperatively, leading to over-preparation. The Ti is one of the effectiveness indexes for preoperative blood preparation, but this index with a cutoff value of 0.5 may be the low threshold to detect the effectiveness of blood utilization because all strategies for preoperative blood preparation had Ti over the cutoff value. The concordant results were similar to what had been shown in previous studies [2,3].  In the pandemic era, the management of limited resources should be taken into consideration since the rate of blood donation has declined globally. Several methods have been proposed to decrease the unnecessary crossmatch in the literature. Palmer et al. [28] proposed a patient-specific blood ordering system to reduce unnecessary crossmatch; this approach had sensitivity, specificity, positive predictive value, and negative predictive value of 41%, 93%, 55%, and 89%, respectively. Lam et al. [29] studied the effectiveness of a prospective physician self-audit transfusion-monitoring system for reducing unnecessary crossmatch. Although several hospitals allow the return of PRC from the ward to the blood bank if not used for transfusion, the time wasted for cross-matched preparation and unnecessary workload should be reduced. In the present study, the ML-based strategy was observed to be an effective approach to mitigate the over-requesting of preoperative blood preparation. This strategy had the lowest C/T ratio, and the highest Tp and Ti when compared with other strategies. Because the predictability of the ML-based tool exhibited an acceptable level of performance, our results are concordant with earlier studies. Chang et al. [30] used ML with various algorithms for predicting blood transfusions in orthopedic surgery; the tool had sensitivity from 69.0-79.2%, specificity from 62.3-71.7%, and accuracy from 70.3-72.2%. Moreover, Huang et al. [31] predicted PRC transfusions in patients with pelvic fracture surgery and reported that an extreme gradient boosting algorithm enabled the best predictability with sensitivity of 93%, specificity of 97%, and accuracy of 95.1%.
In the present study, the benefits of the ML-based strategy were also assessed from the economic perspective. The significant impact of cost savings was observed when the ML-based strategy was compared to the routine-based strategy. The economic benefits were subsequently impacted by the high accuracy performance of the ML-based tool. To the best of the authors' knowledge, the present study is the first to mention the economic impact of the ML approach for MSBOS via a web application, which is one e-health platform for simplifying usage in general practice. However, the limitations of the study should also be discussed. We used local parameters for establishing costs among strategies because we needed to reflect the economic burden from the routine-based strategy in our region. Thus, generalizability is limited. Moreover, external validation of ML in terms of economic impact should be conducted at different times and places to confirm the applicability of the tool and the economic benefits as unnecessary cost savings. In addition, the present study used the same cohort to estimate cost differences among strategies. A randomized controlled trial or prospective observational study with patients assigned to different strategies may be more reflective of the real-world situation than a retrospective study design [32,33].

Conclusion
The ML-based strategy is an effective approach to balance the unnecessary workloads at blood banks and reduce the cost of unnecessary blood products preparation from low C/T ratio, high Tp, and high Ti. Further studies should be performed to confirm the generalizability and applicability of this strategy.